Embodiments of the inventive concept described herein relate to a virtual reality (VR) content sickness evaluating apparatus using a deep learning analysis of a motion mismatch and a method thereof, and more particularly, relate to technologies of analyzing a motion mismatch phenomenon which occurs when a user views VR content based on deep learning and predicting and evaluating sickness.
Recently, VR content has attracted attention as next-generation content like real reality from researchers, industrial worlds, and consumers. Particularly, since there is a high probability that VR content will be used as various applications such as games, broadcasts, entertainments, and training, the scale of related markets is quickly expanded.
VR content may provide a sense of immersion and a sense of reality as if users exist in real image spaces to them by providing a 360-degree image captured as a spherespace. Thus, the user may recognize an image of a wide viewing angle and may move his or her body to look all around a 360-degree image captured as a spherespace.
Herein, as an interest in VR content is more increased, there is much growing concern about viewing safety of users.
For example, when users view VR content with a high sense of immersion, it is known that most users feel cybersickness or VR sickness together with extreme fatigue. Further, when viewers view VR content, it is known that most viewers who feel fatigue or sickness continue having the symptoms for a time after viewing the VR content.
In this case, there are a variety of factors, such as a wide viewing angle, an image distortion, and a low frame rate, as causes which induce VR sickness. One of the most important factors is a mismatch between motion of VR content, that is, simulation motion and real motion of a user.
Hereinafter, a description will be given in detail of a cause which induces VR sickness with reference to FIG. 1.
FIG. 1 is a drawing illustrating an example of VR sickness generated by a mismatch between simulation motion and real motion of a user.
In general, VR content often has fast and diverse motion to provide a vivid sense of immersion to a user. For example, there are a plurality of 360-degree roller coaster images, 360-degree racing images, and the like as VR content. On the other hand, most users sit in chairs or stand while wearing their head-mounted displays (HMDs) and view VR content without large motion.
In this case, there is a mismatch between motion information (or visual recognition information) the user receives with his or her eyes and motion information (or posture recognition information) he or she receives through his or her vestibular organ which recognizes motion and postures of his or her body.
When determining motion of a person's body, a person's brain may finally determine motion through visual recognition information, posture recognition information, and sensory information. However, when a user views VR content having fast and diverse motion, motion recognized through his or her eyes may be very fast or motion recognized through a vestibular organ may be very static. As a result, a brain may feel confusion about motion determination due to a motion mismatch between visual recognition information and posture recognition information, and the user may feel VR sickness due to the confusion.
To address a viewing safety problem of such VR content, there is a need for technologies of analyzing motion of the VR content and motion of the user and automatically evaluating VR sickness in a quantitative manner based on the analyzed motion.
However, according to the related art, there is only a subjective evaluation research in which the user views VR content during a predetermined time and writes a questionnaire about VR sickness or a research of measuring bio-signals while he or she views VR content. Such a subjective evaluation research or a research of measuring the bio-signals needs much time and manpower, and there is a limit in which practicality is very low.
Thus, the inventive concept may propose practical technologies of automatically evaluating VR sickness of VR content to automatically analyze important VR sickness inducing factors such as a motion mismatch between visual recognition information and posture recognition information and predict and evaluate a degree of VR sickness in a quantitative manner based on the important VR sickness inducing factors.